Image recording method and apparatus

ABSTRACT

This invention provides a digital camera having a watermarking function that obviates the need to set various settings again upon setting of an image sensing mode and embedding mode. To this end, an image recording apparatus according to the invention, which includes an image sensing means for sensing an object and a means for embedding predetermined data in image data obtained by the image sensing, comprises a means for setting a first item for defining a mode for the image sensing, and a means for setting a second item for defining a mode for the embedding on the basis of the first item. The image sensing means senses an object on the basis of the first item. The embedding means executes the embedding on the basis of the second item. Alternatively, the apparatus is configured such that the sequence of setting of items for defining an embedding mode and image sensing mode is reversed.

FIELD OF THE INVENTION

[0001] The present invention relates to an image recording apparatus forsensing and recording images and, more particularly, to an imagerecording apparatus capable of recording images sensed by a digitalcamera or the like.

BACKGROUND OF THE INVENTION

[0002] Digital still cameras, which have rapidly become popular in therecent years, are designed to electrically record images sensed by CCDsensors and the like as digital data on recording media such as memorycards, unlike conventional silver-halide cameras designed to form objectimages on the surfaces of films and chemically record the images asanalog images.

[0003] Digital data can be easily processed by computers and easilydistributed through a network and the like. The need for digital stillcameras capable of easily obtaining such digital images therefore areexpected to increase more and more.

[0004] On the other hand, digital data can be easily tampered, e.g.,synthesized, without leaving any sign. Therefore, a problem may arise interms of reliability when a sensed digital image is used as an evidence.Such a problem may not arise often as long as general users enjoyphotography as a hobby. However, a serious problem arises whenphotography is required for some business or legal operation likephotographs as records in a construction site.

[0005] In addition, since digital data can be easily copied/distributed,the copyright on an image cannot be satisfactorily protected.

[0006] Under the circumstances, great expectations are placed on digitalstill cameras capable of improving the reliability of a sensed digitalimage as an evidence and protecting copyright.

[0007] For such a purpose, a technique called “watermark” has beenstudied.

[0008] According to this technique, in digital image/audio data, anotherinformation that is not perceived by a human observer is embedded, andonly a person who owns a proper qualification or right can extract theembedded information. This makes it possible to improve the reliabilityof an image as an evidence or protect copyright.

[0009] The principle of a watermarking technique will be described belowwith reference to Japanese Patent Laid-Open No. 10-290359 and FIG. 2 ina case where digital information is image information (for more detailedinformation, see this reference and Japanese Patent Laid-Open No.10-150517).

[0010]FIG. 2 is a view showing the flow of a procedure for embeddinginformation (embedded information) in image information.

[0011] First of all, an original image (digital image data 101 in FIG.3) is divided into a plurality of blocks each (102 in FIG. 3) consistingof n pixels×m pixels (division processing). Orthogonal transformationsuch as discrete cosine transformation (DCT) is performed for each blockto obtain n×m frequency component matrices (orthogonal transformationprocessing).

[0012] Before information embedding processing, an embedding positionindicating a specific position in the frequency component matricesobtained by orthogonal transformation processing at which to-be-embeddedinformation is embedded is determined by random numbers, and a changeamount indicating how much the value of the corresponding frequencycomponent is changed is determined. The embedding position and changeamount are then acquired/stored as key information.

[0013] When to-be-embedded information is embedded, the information neednot be embedded in all frequency component matrices in one block and maybe embedded across frequency components in a plurality of blocks. Inthis case, a block group with proper contrast in the image is selected.

[0014] By selecting, for example, a low-frequency portion of thefrequency component matrices as an embedding position, information canbe embedded so as not to be perceived by a human observer. In addition,a difference from the original value of a frequency component matrix canbe changed by changing the change amount. This makes it possible tocontrol a deterioration in image quality.

[0015] To-be-embedded information is embedded (embedding processing) bychanging the values of the frequency component matrices in therespective blocks on the basis of the embedding position and changeamount as key information. In addition, images of a plurality of blockseach having n pixels×m pixels is obtained by performing inverseorthogonal transformation for the frequency component matrices of therespective blocks in which the to-be-embedded information is embedded(inverse orthogonal transformation processing). Finally, the images ofthe plurality of blocks obtained by inverse orthogonal transformationprocessing are connected to each other to obtain a watermark image inwhich to-be-embedded information is embedded (reconstructionprocessing).

[0016]FIG. 2 is a view showing the flow of a procedure for extractingembedded information from a watermark image.

[0017] An watermark image is broken up into a plurality of blocks eachconsisting of n pixels×m pixels (segmentation processing). Orthogonaltransformation such as discrete cosine transformation (DCT) is performedfor each divided block to obtain n×m frequency component matrices(orthogonal transformation processing). In addition, an embeddingposition and change amount are obtained from the key information used inthe information embedding processing, and embedded information isextracted from the frequency component matrices of the respective blocks(extraction processing).

[0018] As described above, for example, characteristic features of thewatermarking technique are (1) embedded information cannot be extractedwithout key information used when the information is embedded, (2)embedded information in key information is generated by using randomnumbers, and hence is variable and difficult to decode, (3)to-be-embedded information can be embedded so as not to be perceived bya human observer by using a specific embedding position, and (4) thedegree of deterioration in image quality can be controlled by changingthe change amount.

[0019] The above description is about the “invisible data embedding”method of embedding data that is invisible to a human observer. Incontrast to this, a “visible data embedding” method is also available,in which copyright information or the like is embedded in an originalimage in a perceivable state so as to dissuade the third party fromfraudulently using the image.

[0020] The details of a watermarking technique for such visible data aredisclosed in U.S. Pat. No. 5,530,759 (Japanese Patent Laid-Open No.8-241403).

[0021] According to a conventional camera having a watermarkingfunction, when the user is to change the image sensing modes (imagesensing mode, drive mode, image quality mode, and sensitivity) of thecamera in accordance with the image sensing purpose and object, he/sheoften changes the embedding mode (type mode and image quality mode) ofthe watermarking function and to-be-embedded information at once. Inthis case, in the conventional camera, these settings must be adjustedagain, requiring cumbersome operation.

SUMMARY OF THE INVENTION

[0022] The present invention has been made in consideration of the aboveproblem, and has as its object to provide an image recording method andapparatus in which when the image sensing mode is changed, the embeddingmode and data are automatically set in accordance with the image sensingmode, or when the embedding mode and data are changed, the image sensingmode is automatically set in accordance with the changes.

[0023] In order to achieve the above object, the present invention hasthe following arrangement. There is provided an image recordingapparatus including image sensing means for sensing an object and meansfor embedding predetermined data in image data obtained by the imagesensing, characterized by comprising means for setting a first item fordefining a mode for the image sensing, and means for setting a seconditem for defining a mode for the embedding on the basis of the firstitem, wherein the image sensing means senses an object on the basis ofthe first item, and the embedding means executes the embedding on thebasis of the second item.

[0024] Other features and advantages of the present invention will beapparent from the following description taken in conjunction with theaccompanying drawings, in which like reference characters designate thesame or similar parts throughout the figures thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

[0025]FIG. 1 is a flow chart showing the flow of processing performed ina camera according to the first embodiment of the present invention;

[0026]FIG. 2 is a view showing processing associated with a watermarkingtechnique;

[0027]FIG. 3 is a view showing block division of image data;

[0028]FIG. 4 is a rear view of a digital still camera;

[0029]FIGS. 5A and 5B are views showing display samples on the rearmonitor of the digital still camera;

[0030]FIG. 6 is a top view of the digital still camera;

[0031]FIG. 7 is a view showing an example of the image sensinginformation display liquid crystal unit of the digital still camera;

[0032]FIG. 8 is a block diagram showing the electrical arrangement ofthe digital still camera;

[0033]FIGS. 9A to 9C are views showing monitor display samples and anexample of the image sensing information display liquid crystal unitaccording to the first embodiment of the present invention;

[0034]FIGS. 10A to 10C are views showing monitor display samples and anexample of the image sensing information display liquid crystal unitaccording to the first embodiment of the present invention;

[0035]FIGS. 11A and 11B are views showing monitor display samples and anexample of the image sensing information display liquid crystal unitaccording to the first embodiment of the present invention;

[0036]FIGS. 12A and 12B are views showing monitor display samples and anexample of the image sensing information display liquid crystal unitaccording to the first embodiment of the present invention;

[0037]FIGS. 13A and 13B are views showing monitor display samples and anexample of the image sensing information display liquid crystal unitaccording to the first embodiment of the present invention;

[0038]FIG. 14 is a view showing items to be set in the first embodimentof the present invention;

[0039]FIG. 15 is a view showing items to be set in the second embodimentof the present invention;

[0040]FIG. 16 is a view showing the principle of information embeddingby watermarking;

[0041]FIG. 17 is a view showing the principle of information embeddingby watermarking;

[0042]FIG. 18 is a view showing the principle of information embeddingby watermarking;

[0043]FIG. 19 is a flow chart showing a procedure for processing to beperformed when an image sensing mode is changed;

[0044]FIG. 20 is a flow chart showing a procedure for invisiblewatermarking; and

[0045]FIG. 21 is a flow chart showing a procedure for showing overallprocessing in the embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0046] Embodiments of the present invention will be described in detailbelow with reference to the accompanying drawings.

[0047] [First Embodiment]

[0048]FIG. 4 is a rear view of a digital still camera according to anembodiment of the present invention.

[0049] The digital still camera according to this embodiment has anoptical finder 201. A rear electronic dial 202 for inputting an up-downsignal, a color monitor 203 formed by an LCD for displaying sensedimages and user interface windows, various setting buttons 204, and thelike are arranged on the rear surface portion of the camera.

[0050] With regard to various settings for a watermark, the user caninput/change the corresponding information by using the setting buttons204 and rear electronic dial 202 while watching the menu windowdisplayed on the color monitor 203.

[0051]FIG. 5A shows a display sample 203 of a monitor screen 205 onwhich the watermark embedding mode is set. Referring to FIG. 5A, therespective items are currently set to the contents enclosed in theboxes. According to this setting example, the “embedding” function is“ON”, the “type” of embedding is “invisible” embedding, and theembedding “strength” is “image quality priority”.

[0052] The embedding strength corresponds to the change amount describedabove. When higher priority is to be given to image quality, the changeamount is reduced to decrease the embedding strength to avoid adeterioration in image quality. In contrast to this, when higherpriority is to be given to robustness, the embedding strength isincreased by increasing the change amount. However, this causes adeterioration in image quality. Image quality is in a trade-offrelationship with robustness.

[0053] In this embodiment, when visible watermark information is to beembedded, the information is embedded in part of a corner of image dataobtained by image sensing operation. That is, such information is notembedded in a central portion of the image at which the embeddedinformation interferes with the object image. Visible watermarkembedding processing itself is performed by a known technique.

[0054] When information is to be embedded by invisible watermarkprocessing, the information is embedded in the entire image dataobtained by image sensing operation. Although invisible watermarkembedding processing itself is performed by a known technique, thisembedding is configured to allow the user to choose between increasingwatermark robustness (robustness-priority watermarking) with a slightdeterioration in image quality or minimizing a deterioration in imagequality (image-quality-priority watermarking) with a slight decrease inrobustness. This operation will be described in detail later.

[0055] By operating setting buttons, the display sample on the colormonitor 203 is switched to a display sample 206 for settingto-be-embedded data like that shown in FIG. 5B.

[0056] In this embodiment, “Heizou Hasegawa” is set as “user” data;“1999.06.09 15:37”, as “date/time” data; and “31415926535”, as “cameraID” data, and these data are embedded in sensed image data in watermarkembedding processing.

[0057] The date/time data is information that is held by a conventionalcamera, so necessary data may be referred to from the date/timefunction.

[0058] The camera ID data is a value set in the assembly process in thefactory. This data is unique to the type of camera or maker and cannotbe changed.

[0059] The user therefore can change only the “user” data. Obviously,information that can be arbitrarily added may be added to the aboveinformation. Note that when the user inputs his/her name, he/she shiftsthe current mode to the user name input mode to input letters one by oneby operating the dial 202. The input user name is stored in a flashmemory 306.

[0060]FIG. 6 is a top view of the digital still camera according to thisembodiment. The digital still camera according to this embodimentincludes an image sensing information display liquid crystal unit 211,an upper electronic dial 212 for inputting an up-down signal, varioussetting buttons 213 to 215 for setting image sensing operations for thecamera, a release button 216, and the like.

[0061]FIG. 7 shows an example of the contents displayed on the imagesensing information display liquid crystal unit 211. Reference numeral221 denotes a segment for displaying various image sensing modes, whichinclude “Auto” mode of automatically determining all camera functions,“P” mode of automatically determining a shutter speed and an F-number onthe basis of photometric data, “Tv” mode of automatically determining anF-number (or aperture value) when the user manually sets a shutterspeed, “Av” mode of automatically determining a shutter speed when theuser manually sets an F-number, and “M” mode of manually setting both ashutter speed and an F-number. Only the letter or letters indicating oneimage sensing mode selected by the user is illuminated on the imagesensing information display liquid crystal unit 211.

[0062] Four seven-segment displays 222 display a shutter speed. Twoseven-segment displays 223 arranged on two sides of a dot display anF-number.

[0063] Reference numeral 224 denotes a drive mode display, on which“Sng” indicates the single-exposure mode of image sensing one frame whenthe release button 216 is pressed, and “Cnt” indicates thecontinuous-exposure mode of continuously image sensing frames while therelease button is pressed. One of the character sets is displayed.

[0064] Reference numeral 225 denotes an image quality display fordigital images. In this mode, the user selects an image quality for animage when it is stored in a memory card, i.e., a degree of JPEGcompression is selected. “Fine” indicates a high-image-quality (lowcompression) mode; “Std”, a standard-image-quality (intermediatecompression) mode; and “Eco”, a low-image-quality (high compression)mode.

[0065] Reference numeral 226 denotes a display for setting a sensitivityfor the digital camera, which is used with the seven-segment displays222 to display a film speed converted into the ISO number of aphotographic film. This value depends on the sensitivity of the digitalcamera. Setting this value amounts to adjusting the output gain of theimage sensing sensor to sense an image at a shutter speed and F-numberequivalent to those of a conventional camera loaded with a silver-halidefilm with the set value with respect to the same object. If, forexample, “ISO 800” is set, the gain of the image sensing sensor isadjusted to a relatively high sensitivity in accordance with the setvalue. In this case, however, since the S/N ratio decreases, the imagequality will deteriorate. If a low sensitivity like “ISO 100” is set,the gain of the image sensing sensor is set to a low value, and theimage quality improves.

[0066]FIG. 8 is a block diagram of the digital still camera according tothis embodiment of the present invention.

[0067] A microprocessor 304 in the camera controls various devices inaccordance with the programs stored in the flash memory 306 in advance.

[0068] The microprocessor 304 incorporates a ROM 304 a storing programscorresponding to flow charts to be described later and the table shownin FIG. 14 which will be described in detail later.

[0069] When releasing operation is performed, an object image is formedon an image sensing sensor 301 (e.g., a CCD area sensor), and the imagesignal is A/D-converted by an A/D converter 302. The resultant signal issubjected to color interpolation and filtering in an image signalprocessing IC 303. The resultant data is temporarily stored in a DRAM308 via a data bus 311.

[0070] Note that a watermark table 308 a in which current stateinformation about watermarking is stored is ensured in the DRAM 308(this operation will be described in detail later).

[0071] Digital image data stored in the DRAM 308 is displayed on thecolor monitor 203 as needed.

[0072] Data such as copyright information is embedded in the digitalimage data by a method according to the present invention which will bedescribed later. This data is compressed by a JPEG IC 307. Thecompressed data is then written in a detachable memory card 313 via acard interface (I/F) 310.

[0073] The image data can also be output to a serial bus 312 via aserial I/F 309, and hence can easily be distributed via a network.

[0074]FIG. 1 is a flow chart showing the flow of processing in thedigital camera according to this embodiment of the present invention.

[0075] In the flow chart of FIG. 1, when the release button 216 of thecamera is pressed, the flow advances from step S100 to step S101 todrive the image sensing sensor to perform “integration/read out” (thisstep includes image signal processing and storage of data in the DRAM).

[0076] In step S102, it is checked whether the embedding function is on.If the setting is “ON”, the flow advances to step S103. If the settingis “OFF”, the flow advances to step S104.

[0077] In step S103, to-be-embedded data set in advance (the user name,time, and camera ID in this embodiment) are embedded in the image data.After the processing in step S103, the flow advances to step S104.

[0078] In step S104, the digital image data having undergone theprocessing in step S102 or S103 is stored in the flash memory, and theimage sensing operation is terminated in step S105. Although notspecifically described, JPEG compression is executed before the data isstored in the memory.

[0079]FIG. 14 shows the relationship between the image sensing modes ofthe camera according to the first embodiment and the correspondingwatermark modes. The watermark modes that are automatically set when agiven image sensing mode is set are indicated by “∘”.

[0080] The contents shown in FIG. 14 will be described with reference toFIG. 9 and the subsequent drawings.

[0081] When the image sensing mode of the camera is set to “Auto” modeby using the image sensing mode setting button 213 and upper electronicdial 212 in FIG. 6, the contents shown in FIG. 9A are displayed on thephotographing information display liquid crystal unit 211.

[0082] When “Auto” mode 401 is set, settings are made in modes otherthan the image sensing mode; the drive mode is set to “Sng (single)”402, and the image quality mode is set to “Std (Standard)” 403.

[0083] With this operation, the watermark mode is automatically changedas shown in FIGS. 9B and 9C. The embedding mode is set to “ON” 404; thetype, “Visible” 405; and the strength, “image quality priority” 406. Asto-be-embedded data, “user” 407, “date/time” 408, and “camera ID” 409are set and embedded in the image data. Note that when “Auto” mode isset, the contents shown in FIG. 9B cannot be changed. The reason forthis will be described later.

[0084] When the user sets the image sensing mode and drive mode to “Tv”mode and “Cnt (Continuous)” 410 as shown in FIG. 10A, the contents ofto-be-embedded data are changed to only “user” data 411 shown in FIG.10C in accordance with the setting of this drive mode “Cnt(Continuous)”.

[0085] If the number of to-be-embedded data is large, it takes time toperform embedding processing. The above setting is therefore made toshorten the processing time by limiting the to-be-embedded data to onlyuser data in the continuous-exposure mode, thereby preventing a decreasein the frame speed of the continuous-exposure mode.

[0086] Note that “∘” in FIG. 14 indicates a fixed setting. If, forexample, the user selects “Auto” as an image sensing mode, “embedding”is ON. An image-quality-priority visible watermarking method is set asan embedding method, and “user+image sensing date+camera ID” is embeddedas to-be-embedded data. These settings cannot be changed. Note that ifthe user selects “Auto”, “Std” is set as image quality for storage.

[0087] Note that “-” indicates no setting can be selected. If, forexample, “Fine” is selected (an image sensing mode other than “Auto” canbe selected), “embedding” is OFF. Since the user can neither choosebetween visible watermarking and invisible watermarking nor select animage quality based on watermarking and the type of data to be embedded,these items are indicated by “-” as shown in FIG. 14.

[0088] The user can arbitrarily select settings in the remaining itemswith no symbols. If, for example, the user selects the manual mode “M”as an image sensing mode, he/she can arbitrarily choose to embed data ornot and can arbitrarily choose between the image-quality-priority modeand the robustness-priority mode when data is to be embedded. Even if,however, the user selects the manual mode “M”, watermarking cannot beperformed when the storage form is set to “Fine”. If “embedding” is setto OFF, the user cannot select data such as the type of data to beembedded.

[0089] As described above, the image sensing modes of the camera of thisembodiment include the five types of modes, i.e., “Auto”, “P”, “Tv”,“Av”, and “M”. The user can select one of them by operating the upperelectronic dial 212. When one image sensing mode is selected,information about watermarking in the corresponding image sensing modeis read out from the table 304 a (see FIG. 14) and stored in thewatermark table 308 a in the DRAM 308. If, for example, “Auto” mode isselected, the data shown in FIG. 18 are stored in the watermark table308 a. As shown in FIG. 18, “embedding” is set to ON, visiblewatermarking is performed (visible watermark information is embedded ina corner of an image), and image data is stored as high-image-qualitydata. In addition, the user name, image sensing date, and camera ID areall stored as to-be-embedded data. In “Auto” mode, flags indicating thatthese parameters associated with watermarking cannot be changed are set,and hence the user cannot change them. When the parameters are to bechanged, the user must select another image sensing mode and update thewatermark table 308 a.

[0090] When the user sets the image quality (JPEG compression) mode to“Fine (high image quality)” mode 412, as shown in FIG. 11A, theembedding mode is turned off (413 in FIG. 11B).

[0091] The watermarking technique is a procedure for embeddinginformation by changing image data itself, and hence may affect theimage quality more or less. For this reason, when the user sets theimage quality mode to “Fine (high image quality)” mode, the watermarkingfunction is turned off.

[0092] Likewise, when the user sets the image quality (JPEG compression)mode to “Eco (low image quality)” mode 414, as shown in FIG. 12A,“strength” in the watermark mode is changed to “robustness priority”(415 in FIG. 12B).

[0093] When the user sets the image quality mode to “Eco (low imagequality)” mode, since high priority is not given to image quality, theembedding “strength” of watermark information may be set to “robustnesspriority” rather than “image quality priority”.

[0094] When the user sets “800” (417) equivalent to ISO 800 (418) of aphotographic film, as shown in FIG. 13A, since this indicates a casewhere the sensitivity is to be increased with respect to a dark object,the S/N ratio of image data greatly decreases. In such setting,therefore, “strength” in the watermark mode may be changed to“robustness priority” (419 in FIG. 13B).

[0095] Watermarking used in this embodiment will be described next.Robustness-priority watermarking and image-quality-priority watermarkingwill be described afterward.

[0096] This embodiment uses the principle of a method called thePatchwork method to embed additional information (the user name, imagesensing date, and camera ID in this embodiment). The Patchwork method isdisclosed in, for example, Walter Bender, Daniel Gruhl, NorishigeMorimoto, and Anthony Lu, “Techniques for data hiding (First Volume)”,NIKKEI ELECTRONICS, 1997.2.24. The principle of the Patchwork methodwill be described first.

[0097] The principle of the Patchwork method will be described withreference to FIG. 16. Referring to FIG. 16, two subsets A and B are setin an image. Assume that the subset A consists of a plurality of subsetelements represented by a subset a_(i) 501, and the subset B consists ofa plurality of subset elements represented by a subset b_(i) 502.

[0098] If these two types of subset elements do not overlap each other,additional information can be embedded by the Patchwork method in thisembodiment.

[0099] Assume that the subsets A and B are sets each consisting of Nelements represented by A={a₁, a₂, . . . , a_(N)} or B={b₁, b₂, . . . ,b_(N)}. Assume also that each of elements a_(i) and b_(i) of the subsetsA and B represents a pixel having a pixel value or a pixel set.

[0100] In this case, an index d is defined as:

d=1/N·Σ(a ₁ −b _(i))

[0101] where Σ is the sum of i=1 to N.

[0102] This index indicates the expectation of the difference betweenthe pixel values of two sets.

[0103] Assume that proper subsets A and B are selected for a generalnatural image, and the index d is defined. In this case, if N is asufficiently large value, then the index d tends to become

d≈0

[0104] The index d therefore exhibits a distribution like a distribution601 in FIG. 17. This value d will be referred to as a reliabilitydistance hereinafter.

[0105] If, for example, bit information “1” is to be embedded as eachbit of additional information, the following operation(addition/subtraction) is performed:

a′ _(i) =a _(i) +c

b′ _(i) =b _(i) −c

[0106] In this operation, “c” is added to the pixel value of everyelement of the subset A, and “c” is subtracted from the pixel value ofevery element of the subset B. In this embodiment, this value “c” willbe referred to as “embedding depth” hereinafter.

[0107] In this case, as in the above case, the subsets A and B areselected from the image in which additional information is embedded, andthe index d is calculated to obtain (each Σ is the sum total of i=1 toN): $\begin{matrix}{d = \quad {{1/N} \cdot {\sum\left( {a_{i} - b_{i}} \right)}}} \\{= \quad {{1/N} \cdot {\sum\left\{ {\left( {a_{i} + c} \right) - \left( {b_{i} - c} \right)} \right\}}}} \\{= \quad {{1/N} \cdot {\sum\left\{ {\left( {a_{i} - b_{i}} \right) + {2c}} \right\}}}} \\{\approx \quad {2\quad c}}\end{matrix}$

[0108] That is, d takes a value separated from 0 by a predetermineddistance (=2c), and exhibits a distribution 602 in FIG. 17.

[0109] When bit information (bit information “0”) is to be embedded, thefollowing operation is performed: a_(i)^(′) = a − cb_(i)^(′) = b_(i) + c

[0110] Then, the reliability distance d becomes: $\begin{matrix}{d = \quad {{1/N} \cdot {\sum\left( {a_{i} - b_{i}} \right)}}} \\{= \quad {{1/N} \cdot {\sum\left\{ {\left( {a_{i} + c} \right) - \left( {b_{i} + c} \right)} \right\}}}} \\{= \quad {{1/N} \cdot {\sum\left\{ {\left( {a_{i} - b_{i}} \right) - {2c}} \right\}}}} \\{\approx \quad {{- 2}\quad c}}\end{matrix}$

[0111] As a consequence, d takes a value separated from 0 by apredetermined distance (=−2c) in the negative direction, as indicated byreference numeral 603 in FIG. 17.

[0112] That is, for a given image, whether additional information isembedded can be determined by calculating the reliability distance d forthe image.

[0113] If reliability distance d≈0, no additional information isembedded. If the reliability distance d is a positive value separatedfrom 0 by a predetermined amount (threshold) or more, it can bedetermined that bit information “1” is embedded. If the reliabilitydistance d is a negative value separated from 0 by a predeterminedamount, it can be determined that bit information “0” is embedded.

[0114] In this embodiment, since the user name, image sensing date, andcamera ID are to be embedded, a plurality of bits must be embedded.

[0115] If, for example, a total of Q bits are to be embedded, the imagemay be divided into M areas (M≧Q), and the bits may be embedded in therespective areas. Each of the areas obtained by dividing the image isdefined as a pixel block, each pixel block contains N pixels (N is aneven number, for example), and the respective pixels are defined as X₁,X₂, . . . , X_(N.)

[0116] In this case, the odd-numbered pixels X₁, X₃, . . . , X_(N) _(¹)may constitute the subset A, and even-numbered pixels X₂, X₄, . . . ,X_(N) may constitute the subset B.

[0117] When embedded bit information is to be determined from thereliability distance d in this manner, information can be extracted withstatistically sufficient reliability by setting a proper thresholdbetween 0 and reliability distance 2c and determining the presence ofembedded information when the absolute value of the reliability distanceis larger than the threshold.

[0118] Letting σ be the standard deviation of the normal distribution601, if no additional information is embedded, the reliability distanced appears in the interval between −1.96σ to +1.96σ (95% reliabilityinterval) indicated by the hatched portion in FIG. 6 with a 95%probability.

[0119] If, therefore, the threshold is increased, the probability thatthe reliability distance d appears outside the threshold decreases. Thismakes it possible to extract information with high reliability.

[0120] The embedding depth “c” is added/subtracted to/from each pixelvalue. If, therefore, the embedding depth “c” is subtracted from a pixelvalue less than “c”, the pixel value takes a negative value. Assume thatthe maximum value of a pixel is 255 when it is expressed by eight bits.In this case, if “c” is added to a pixel value larger than 255−c, theresultant value becomes larger than the maximum value. Therefore, pixelsin which information can be actually embedded are pixels whose pixelvalues P satisfy c<P<255−c. As a consequence, the number of pixels inwhich information can be embedded decreases or may decrease. However,since the number of image sensing elements of the CCD of a currentlyavailable digital camera is very large, pixels in which information canbe embedded with sufficient precision can be ensured.

[0121] If the embedding depth “c” is increased, the normal distributions602 and 603 are separated from the distribution 601 to allow an increasein threshold. If, however, the embedding depth “c” is set to a smallvalue, although a deterioration in image quality is small, the precisionof determination on the presence/absence of embedded informationdecreases. In contrast to this, if the embedding depth “c” is set to alarge value, although the precision of determination on thepresence/absence of embedded information increases, the image qualitydeteriorates.

[0122] In this embodiment, when “image-quality-priority watermarking” isselected as an invisible watermark embedding mode in FIG. 14, anembedding depth “c1” is used; when “robustness-priority watermarking” isselected, an embedding depth “c2” is used. Obviously, in this case, therelationship between c1 and c2 satisfies c1<c2.

[0123] The above description is about an example ofimage-quality-priority watermarking/robustness-priority watermarking.Obviously, this embodiment may use another method. This is because theessential characteristic feature of the embodiment is that whetherwatermarking is performed is determined in accordance with the selectionof an image sensing mode, and/or whether visible watermarking orinvisible watermarking is performed is determined in accordance with theselection if watermarking is performed, and/or whetherrobustness-priority watermarking or image-quality-priority watermarkingis performed is determined in accordance with the selection.

[0124] A procedure for processing performed by the camera according tothis embodiment will be described in more detail with reference to theflow chart of FIG. 21.

[0125] In step S401, it is checked whether the release button 216 ispressed. If NO in step S401, the flow advances to step S402 to performprocessing corresponding to operation other than the pressing of therelease button 216. The processing to be performed in step S402 includesselection of an image sensing mode.

[0126] In this case, for example, the selection of an image sensing modemay be performed in accordance with the procedure shown in FIG. 19.

[0127] Steps S201 to 205 are decision processing to be performed when animage sensing mode is selected with the dial 212. If “Auto” mode isselected, data indicating whether embedding is performed, dataindicating whether visible or invisible watermarking is performed, dataindicating whether image-quality-priority watermarking orrobustness-priority watermarking is performed, if invisible watermarkingis to be performed, and data indicating the type of target to beembedded, which are parameters associated with watermarking in “Auto”mode in the table 304 a (see FIG. 14) in the ROM 304, are written in thewatermark table 308 a in the DRAM 308. In this case, as shown in FIG.14, flags indicating inhibition of changes are set with respect to theitems to which “∘” or “-” is attached.

[0128] Operation similar to that described above is performed when “P”mode is selected. However, as shown in FIG. 14, since data indicatingwhether embedding is to be performed can be changed, as the data in thetable 308 a, the data selected in the immediately preceding imagesensing mode is left. Note, however, all the items are set as changeableitems.

[0129] The above processing is a procedure associated with the operationof the dial 212. If, for example, the user operates various switches,after selection of “P” mode, to determine whether to perform embedding,whether to perform visible or invisible watermarking if embedding isdetermined, and whether to perform image-quality-priority watermarkingor robustness-priority watermarking if invisible watermarking isdetermined, the contents are reflected in the table 308 a.

[0130] In the above manner, the user makes settings for image sensingmodes and watermarking.

[0131] Referring back to FIG. 21, information about watermarking isstored in the table 308 a or updated in the above manner, and imagesensing processing is performed in step S403 when the release button 216is pressed. The image data obtained by image sensing is stored in theDRAM 308, as described above.

[0132] When image data is completely stored in the DRAM 308, the flowadvances to step S404 to check whether to perform watermarking. Thisdecision processing is performed by checking whether “Watermarking” inthe table 308 a is ON. If information is not embedded by watermarking,the flow advances to step S408 to compress/encode the data in the modeset by the user. The resultant data is stored in the memory card 313 instep S409, and this processing is terminated.

[0133] If it is determined that the setting indicates the execution ofwatermarking, the flow advances to step S404 to check by looking up thetable 308 a whether to perform visible watermarking or invisiblewatermarking. If it is determined that visible watermarking isperformed, target items (to be embedded) are merged, and the resultantdata is embedded in part of a corner of the image stored in the DRAM 308by visible watermarking. Visible watermarking may be performed by thetechnique disclosed in U.S. Pat. No. 5,530,759, and a descriptionthereof will be omitted. When this embedding operation is complete, theresultant data is stored in the memory card 313 through steps S408 andS409.

[0134] If it is determined in step S405 that invisible watermarking isto be performed, the information is embedded by invisible watermarkingin step S407.

[0135] The processing in step S407 may be performed in accordance withthe flow chart of FIG. 20.

[0136] In step S301, the watermark table 308 a is looked up to determinewhich items are selected as target items to be embedded at this time,and the target items are merged to generate to-be-embedded information.If image sensing time information is set as a target to be embedded, thecurrent time is read from a timer 314.

[0137] It is checked in step S302 whether the information is to beembedded by image-quality-priority watermarking or robustness-prioritywatermarking. This decision processing is performed by checking“embedding level” in FIG. 18.

[0138] If it is determined that the information is to be embedded byimage-quality-priority watermarking, the flow advances to step S303 tosubstitute the value c1 for the embedding depth c. If it is determinedthat the information is to be embedded by robustness-prioritywatermarking, the value c2 is substituted for the embedding depth c instep S304. In this case, c1 and c2 satisfy c1<c2, as described above.

[0139] When the embedding depth c is determined in this manner, the flowadvances to step S305 to embed one bit of the to-be-embedded informationin the image. The processing in step S305 is repeated until it isdetermined in step S306 that all the bits are embedded.

[0140] When all the bits are embedded, this processing is terminated.The flow then advances to steps S408 and S409 in FIG. 21 tocompress/encode the image data. The resultant data is then stored in thememory card 313.

[0141] As described above, according to this embodiment, when the userof the camera changes the image sensing mode of the camera in accordancewith an image sensing purpose and object, the watermark mode andto-be-embedded data are automatically changed accordingly. This makes itpossible to set an optimal watermark mode in accordance with the imagesensing purpose and object without making various setting again, thusgreatly improving operability.

[0142] [Second Embodiment]

[0143] In the first embodiment, when the image sensing mode of thecamera is changed, the corresponding watermark mode is changedaccordingly. In contrast to this, as shown in FIG. 15, a camera may beconfigured such that when the watermark mode is changed, thecorresponding image sensing mode is automatically changed accordingly.

[0144] More specifically, when the “image quality (strength) mode” ofwatermarking is set to “image quality priority”, even if the “imagequality (compression) mode” of the image sensing mode of the camera isset to “Std (standard)” or “Eco (low image quality)”, the mode ischanged to “Fine (high image quality)” in accordance with the watermarkmode.

[0145] In this case, “sensitivity” is also changed to “low sensitivity”in accordance with the watermark mode to increase the S/N ratio of theimage.

[0146] If “to-be-embedded data” is set to “user”, “date”, and “cameraID”, it takes much time for embedding processing during releasingoperation. Even if, therefore, the “drive mode” of the camera is set to“Cnt (continuous)”, this mode is changed to “Sng (single)” because it isdifficult to maintain a proper frame speed.

[0147] As described above, according to this embodiment, when the userof the camera changes the watermark mode, the image sensing mode of thecamera is changed accordingly. This makes it possible to set an optimalimage sensing mode without making various settings again, thus greatlyimproving operability.

[0148] In the above embodiment, the timing of embedding of informationby watermarking is set in accordance with image data before DCT.However, this timing may be set in accordance with each frequencycomponent after DCT. As is obvious, when several levels of robustness ofwatermarking are to be set, in particular, a method other than thePatchwork method may be used. It is essential that the camera has thefunction of properly determining, in accordance with image sensing modesand the like or storage mode, whether to perform watermarking, whichkind of watermarking is to be used, if it is determined thatwatermarking is performed, and/or whether to perform robustness-prioritywatermarking or image-quality-priority watermarking.

[0149] The present invention may be applied to a system constituted by aplurality of devices (e.g., a host computer, an interface device, areader, a printer, and the like) or an apparatus comprising a singledevice (e.g., a copying machine, a facsimile apparatus, or the like).

[0150] The object of the present invention is realized even by supplyinga storage medium storing software program codes for realizing thefunctions of the above-described embodiments to a system or apparatus,and causing the computer (or a CPU or an MPU) of the system or apparatusto read out and execute the program codes stored in the storage medium.In this case, the program codes read out from the storage medium realizethe functions of the above-described embodiments by themselves, and thestorage medium storing the program codes constitutes the presentinvention. The functions of the above-described embodiments are realizednot only when the readout program codes are executed by the computer butalso when the OS (Operating System) running on the computer performspart or all of actual processing on the basis of the instructions of theprogram codes.

[0151] The functions of the above-described embodiments are alsorealized when the program codes read out from the storage medium arewritten in the memory of a function expansion card inserted into thecomputer or a function expansion unit connected to the computer, and theCPU of the function expansion card or function expansion unit performspart or all of actual processing on the basis of the instructions of theprogram codes.

[0152] When the present invention is applied to the above storagemedium, program codes corresponding to the flow charts described aboveare stored in the storage medium.

[0153] As has been described above, according to the present invention,when the user of the camera changes the image sensing mode of the camerain accordance with an image sensing purpose and object, the embeddingmode of the watermarking function and to-be-embedded data are changedaccordingly. Alternatively, the image sensing mode of the camera ischanged in accordance with the change of the watermark mode. This makesit possible to automatically set an optimal image sensing mode orwatermark mode in accordance with the image sensing purpose and object,thus greatly improving operability.

[0154] As many apparently widely different embodiments of the presentinvention can be made without departing from the spirit and scopethereof, it is to be understood that the invention is not limited to thespecific embodiments thereof except as defined in the appended claims.

What is claimed is:
 1. An image recording apparatus including image sensing means for sensing an object and means for embedding predetermined data in image data obtained by the image sensing, comprising: means for setting a first item for defining a mode for the image sensing; and means for setting a second item for defining a mode for the embedding on the basis of the first item, wherein said image sensing means senses an object on the basis of the first item, and said embedding means executes the embedding on the basis of the second item.
 2. An image recording apparatus including image sensing means for sensing an object and means for embedding predetermined data in image data obtained by the image sensing, comprising: means for setting a third item for defining a mode for the embedding; and means for setting a fourth item for defining a mode for the image sensing on the basis of the third item, wherein said image sensing means senses an object on the basis of the fourth item, and said embedding means executes the embedding on the basis of the third item.
 3. The apparatus according to claim 1, wherein the first or fourth item defines values associated with an exposure time and aperture of said apparatus.
 4. The apparatus according to claim 1, wherein the first or fourth item defines a value associated with a continuous-exposure frame count of said apparatus.
 5. The apparatus according to claim 1, wherein the first or fourth item defines a value associated with image quality of a sensed image.
 6. The apparatus according to claim 1, wherein the first or fourth item defines a value associated with sensitivity with respect to an amount of light received.
 7. The apparatus according to claim 1, wherein the second or third item defines a type of watermarking represented by the predetermined data to be embedded.
 8. The apparatus according to claim 1, wherein the second or third item defines a value associated with an embedding strength of the predetermined data.
 9. The apparatus according to claim 1, wherein the second or third item defines a type of the predetermined data to be embedded.
 10. An image recording method including the image sensing step of sensing an object and the step of embedding predetermined data in image data obtained by the image sensing, comprising: the step of setting a first item for defining a mode for the image sensing; and the step of setting a second item for defining a mode for the embedding on the basis of the first item, wherein the image sensing step comprises sensing an object on the basis of the first item, and the embedding step comprises executing the embedding on the basis of the second item.
 11. An image recording method including the image sensing step of sensing an object and the step of embedding predetermined data in image data obtained by the image sensing, comprising: the step of setting a third item for defining a mode for the embedding; and the step of setting a fourth item for defining a mode for the image sensing on the basis of the third item, wherein the image sensing step comprises sensing an object on the basis of the fourth item, and the embedding step comprises executing the embedding on the basis of the third item.
 12. The method according to claim 10, wherein the first or fourth item defines values associated with an exposure time and aperture of said image recording apparatus.
 13. The method according to claim 10, wherein the first or fourth item defines a value associated with a continuous-exposure frame count of said image recording apparatus.
 14. The method according to claim 10, wherein the first or fourth item defines a value associated with image quality of a sensed image.
 15. The method according to claim 10, wherein the first or fourth item defines a value associated with sensitivity with respect to an amount of light received.
 16. The method according to claim 10, wherein the second or third item defines a type of watermarking represented by the predetermined data to be embedded.
 17. The method according to claim 10, wherein the second or third item defines a value associated with an embedding strength of the predetermined data.
 18. The method according to claim 10, wherein the second or third item defines a type of the predetermined data to be embedded.
 19. A computer-readable memory storing a code for executing the image sensing step of sensing an object and a code for executing the step of embedding predetermined data in image data obtained by the image sensing, comprising: a code for executing the step of setting a first item for defining a mode for the image sensing; and a code for executing the step of setting a second item for defining a mode for the embedding on the basis of the first item, wherein the code for executing the image sensing step comprises sensing an object on the basis of the first item, and the code for executing the embedding step comprises executing the embedding on the basis of the second item.
 20. A computer-readable memory storing a code for executing the image sensing step of sensing an object and a code for executing the step of embedding predetermined data in image data obtained by the image sensing, comprising: a code for executing the step of setting a third item for defining a mode for the embedding; and a code for executing the step of setting a fourth item for defining a mode for the image sensing on the basis of the third item, wherein the code for executing the image sensing step comprises sensing an object on the basis of the fourth item, and the code for executing the embedding step comprises executing the embedding on the basis of the third item.
 21. An image recording apparatus having image sensing means, comprising: selection means for selecting one of a plurality of image sensing modes; embedding means for embedding information as a watermark in an image; determination means for determining, in accordance with the image sensing mode selected by said selection means, whether to activate said embedding means; and control means for, when said determination means determines that the information is to be embedded, performing control to activate said embedding means to embed the information in the image data sensed by said image sensing means.
 22. The apparatus according to claim 21, wherein the information includes information specifying a user name, image sensing date, and image recording apparatus.
 23. The apparatus according to claim 21, wherein said embedding means comprises first embedding means for embedding information as a visible watermark in an image, and second embedding means for embedding information as an invisible watermark in an image, and said determination means comprises means for determining one of said first and second embedding means when embedding is to be performed.
 24. The apparatus according to claim 21, wherein said embedding means comprises first embedding means for embedding information with priority given to image quality of an image in which the information is to be embedded, and second embedding means for embedding information with priority given to robustness of the information to be embedded, and means for determining one of said first and second embedding means when information is to be embedded.
 25. The apparatus according to claim 21, wherein said embedding means comprises first embedding means for embedding information as a visible watermark in an image, second embedding means for embedding information as an invisible watermark in an image with priority given to image quality of the image in which the information is to be embedded, and third embedding means for embedding information as an invisible watermark in an image with priority given to robustness of the information to be embedded, and said determination means comprises means for determining one of said first to third embedding means when embedding is to be performed.
 26. The apparatus according to claim 21, wherein said determination means determines, in accordance with image quality set when a sensed image is stored in a predetermined storage medium, whether to perform embedding.
 27. A control method for an image recording apparatus having image sensing means, comprising: the selection step of selecting one of a plurality of image sensing modes; the embedding step of embedding information as a watermark in an image; the determination step of determining, in accordance with the image sensing mode selected in the selection step, whether to activate the embedding step; and the control step of, when it is determined in the determination step that the information is to be embedded, performing control to activate the embedding step to embed the information in the image data sensed in the image sensing step. 